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Background/context: Current national educational policy embodied by the No Child Left 
Behind Act requires that all students, including English language learners, meet high standards in 
language arts, as well as science and math. While expectations for content area achievement are 
high, findings from recent international and national studies underscore the importance of 
improving the literacy and science achievement of middle grades students, including English- 
language learners (ELLs) (Rampey, Dion, & Donahue, 2009; National Center for Education 
Statistics, n.d ). While the need is great to improve instruction for these students, the research 
base is very thin compared with that for English proficient students (August & Shanahan, 2008). 
Our goal in this paper was to address the need for research in this area by examining the effects 
of an enhanced science curriculum designed specifically for students who were English-language 
learners (ELLs) 

Purpose / objective / research question / focus of study: The goal of this study was to assess 
the effectiveness of an intervention— Quality English and Science Teaching (QuEST)— designed 
to develop the science knowledge and academic language of middle grades English language 
learners studying science in their second language and their English-proficient classmates. Both 
English language learners and English-proficient students are incorporated in our study design, 
and we have tested explicitly for interactions of language status with experimental and traditional 
forms of instruction. An overriding principle in our research was to make science instruction 
effective for both English language learners and English-proficient students because these two 
groups of students are often together in the same classrooms in the middle grades. Thus, the 
interventions we developed used what we know about high-quality science instruction for 
students in the middle grades as a starting point but made modifications to ensure ELLs 
understood the content of the lessons delivered in English and concurrently developed their 
English proficiency. For this reason, we also drew on research about the role of English language 
proficiency, learning in a second language, and knowledge acquired in the first language, in this 
case Spanish, to tailor the intervention to meet the language and literacy needs of English 
language learners. 

Setting: Five middle schools located in a large, high poverty district in the Rio Grande Valley 
with a high percentage of Latino English-language learners. 

Population / Participants / Subjects: Participants in this study included 890 students; 562 were 
English-language-learners and 328 were English proficient. The research involved forty sections 1 
of students receiving their science instruction from one of ten teachers at the five middle schools. 
Each teacher contributed four sections, two of which were randomly assigned to treatment and 
two of which were assigned to typical instruction. Class sizes of the four sections for each 
teacher were comparable in size within and between teachers, ranging from a minimum of 1 1 to 
13 to a maximum of 29 to 32 for treatment and control, respectively 

Intervention / Program / Practice: In the district, the science curriculum at the sixth grade level 
consisted of the Prentice Hall textbook and workbook, and district developed labs aligned with 
text book content and state and district standards. The intervention, called Project QuEST, was 
comprised of two components that were not present in the district at the time of the intervention: 



1 At the middle grades level, a section is a classroom of students. In this district, middle grades teachers were 
generally assigned six sections of students a day. 
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instructional materials and professional development. Both components sought to ensure that 
both English-language learners and English proficient students acquired high levels of science 
knowledge and concurrently developed language and literacy skills. District and state standards 
were used to set science, language, and literacy goals. The intervention materials included a 
teacher guide and instructional charts, a student guide and instructional charts, and supplies for 
hands-on science activities. The intervention materials and instructional practices built on a 
highly rated inquiry approach to teaching science to monolingual English speakers developed by 
the Biological Science Curriculum Study (BSCS) 2 The approach puts a premium on hands-on 
experimentation that aids students in building their own understanding of new concepts. 

Additionally, the intervention required that teachers use scaffolding techniques that 
previous research has shown to foster English language learners’ understanding of academic 
content and has enhanced their language and literacy development (August & Shanahan, 2008). 
First, visuals were consistently used in science lessons, including illustrations of vocabulary 
concepts and graphic organizers. Second, students were given a preview of the activities they 
would be conduct to ensure that they understood the goals and procedures. Third, explicit 
vocabulary instruction of both general and discipline-specific vocabulary was emphasized. 
Fourth, ELLs were paired with English proficient students who served as language models. 
Professional development included three training sessions and ongoing weekly mentoring. 
Teachers learned how encourage expression of students’ own ideas, build upon information 
students provided and experiences they have had, and guide students to increasingly 
sophisticated levels of understanding and language. 

Research Design: The intervention was implemented in 20 sixth grade science classrooms 
taught by 10 teachers in five middle schools, while the districts standard curriculum was 
implemented in 20 sixth grade science classrooms taught by these same 10 teachers. Teachers 
taught two of their science sections using the QUEST materials and strategies for a period of 9 
weeks, and taught two other sections of science using the district standard curriculum over the 
same time period. For each teacher, the four sections were randomly assigned to Quest and to the 
district curriculum so that each teacher taught two sections under each condition. The sample of 
students included English-language learners, former English-language learners, and fluent 
English-speaking students. 

Treatment effects were tested separately for science knowledge and vocabulary using 
analysis of covariance, with the analogous pre-test serving as the covariate. Analyses included 
fixed effects of treatment assignment and the covariate, and random effects for section and 
teacher. Treatment effects were tested at the level of the section, which was the unit of 
assignment. All models were fit in HLM 6.06 (Raudenbush, Bryk, & Congdon, 2008). 

Data Collection and Analysis: All students in the 40 science sections were assessed using the 
vocabulary and passage comprehension subtests of the GRADE assessment, form A, level M 
prior to the onset of treatment as a safeguard against possible unhappy randomization. In 
addition, students were assessed in English for science and vocabulary knowledge using 
researcher-developed assessments aligned with the district’s sixth grade science curriculum 
covered during the nine weeks the intervention was in place, namely Living Systems and the 



2 

~ BSCS is a nonprofit corporation that endeavors to improve all students’ understanding of science and technology 
by developing exemplary curricular materials, supporting their widespread and effective use, providing professional 
development, and conducting research and evaluation studies. 
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Environment. The items for the science tests were selected from items released from past state 
science tests as well as from an item bank available from the publisher of the text book both 
treatment and control classrooms were using. The researcher developed vocabulary measure 
consisted of both the general academic and discipline specific words that appeared in the 
textbook. Reliability estimates based on coefficient alpha indicate that the assessments 
functioned as desired. A fidelity/quality of instruction observation instrument was developed by 
the research team and used to document whether and how well the specific components of the 
intervention were being implemented in the treatment classrooms, the activities that were taking 
place in the control classrooms, and the general quality of classroom instruction. The same 
instrument was used in both treatment and control classrooms. All student assessments were 
group administered by trained research assistants prior to or following each curricular unit. Each 
teacher was observed teaching treatment and control sections by trained observers for the sake of 
rating fidelity to treatment and quality of instruction. 

To examine the effects of treatment in this cluster-randomized trial, we fit separate three- 
level, multi-level analysis of covariance models for the vocabulary and science outcomes, using 
the corresponding pre-test as the covariate. We also examined evidence for treatment differences 
on the pre-tests using a three-level analysis of variance model. In both the analysis of pre-tests 
and post-tests, the levels of the model corresponded to students, sections, and teachers, with 
students nested within sections, and sections nested within teacher, and treatment assigned at the 
section level. Thus, the effects of treatment are tested within teacher and averaged across 
teachers. In this sense, treatment is a repeated, or “within subjects,” factor when viewed from 
the teacher level, and a “between subjects” factor when viewed from the section level. The 
intercept was allowed to vary randomly at each level. 

Findings / Results: On average, the QuEST intervention produced positive gains in performance 
for students, regardless of their status as English Language Learners or native speakers of 
English. In addition, the QuEST intervention produced positive effects for both Science and 
Vocabulary outcomes for students as reflected in the curriculum based measures, which reflected 
the material being taught in both the treatment and control sections. Standardized effect sizes for 
the covariate adjusted means were in the small to moderate range for Science (.15 to .24) for the 
entire sample, as well as for the ELL students when examined separately (g = .16 to .25), 
depending on whether pre-test or post-test standard deviation is used as the baseline for the effect 
size computation. Results for vocabulary were somewhat more favorable, but still in this same 
general range, although more toward the moderate end ( g = .28 to .37) for the entire sample, and 
for the ELL students alone (g = .26 to .37). To put these effect sizes in perspective, at g = .25, if 
schools could achieve this return for each of the three years of middle school, the net effect over 
three years would be a gain of .95 standard deviation units (1.25 3 ), or almost a full standard 
deviation of improvement over treatment as usual. These gains would be added to the annual 
gains due to instruction, which in the present study produced gains of only. 27 standard 
deviations for Vocabulary for native speakers of English, and gains of .75 standard deviation 
units for Vocabulary for ELL students. Annual Science gains were somewhat larger, namely .88 
standard deviation units for native speakers of English, and 1.01 standard deviation units for ELL 
students. These gains are estimated from the change observed in students in the control sections 
relative to the pre-test standard deviation. Thus, treatment gains in Vocabulary from three years 
of QuEST would be roughly double what native speakers of English would be expected to gain 
over that same time frame in traditional instruction, while ELL students would gain better than 
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four years of vocabulary growth in three years. Both native speakers of English and ELL 
students would gain in three years what might have been expected over four years of traditional 
instruction. These estimates of gain assume only that the effect size of .25 could be maintained in 
each of the three years of Middle School instruction. That is, they assume no compounding of 
content are or vocabulary knowledge as students acquire more knowledge. In so far as effect 
sizes for QuEST were comparable for ELL and native English speakers, this assumption appears 
reasonable for the time being, although one could easily argue that gains in language should lead 
to a compounding of effects on the acquisition of other language skills. If the QuEST strategies 
could be adapted to other content areas with similar efficacy, one could obviously expect a 
compounding effect with respect to content area knowledge and vocabulary gains in so far as 
effects would be observed across multiple domains of knowledge acquisition, and words learned 
in one content area that have general purpose utility, would obviously transfer to other content 
areas and not need to be relearned in that other context. Whether such gains are possible and 
could be obtained across multiple content areas is a matter of speculation at this point. However, 
it seems clear, that gains of .25 standard deviations, if sustainable year over year can result in a 
very significant shift in the achievement and language distributions for both ELL students and 
students who are native speakers of English. 

Conclusions: Overall, it appears that the implementation of QuEST improves the quality of 
teachers’ science instruction and raises student performance on curriculum based measures of 
Vocabulary and Science. Project QuEST differed from the practices in the control classrooms by 
making alterations to accommodate the needs of ELLs and build on their strengths. Consistent 
with the literature on effective L2 instruction, the content was made clear to students through the 
use of visuals, modeling, and ongoing discussion. Additionally, students’ English oral 
proficiency was developed in the context of science instruction through explicit vocabulary 
instruction, guided reading, and partnering with classmates who were more English proficient. 

Project QuEST makes an important contribution to the field in that there is very little 
research that explores whether enhancements to traditional practices are necessary or improve 
the traditional versions, and importantly, whether modifications to traditional practices to make 
them more effective with English-language learners also make them more effective with 
monolingual English students. To be optimal, ESL-enhanced instructional practices must 
enhance the learning of English-language learners in the classroom AND must be no less 
effective than traditional methods of instruction for monolingual English students. Because ELL 
students are often placed in classrooms with native speakers of English it is critical that the 
development of instructional methods to specifically benefit the EL students in mixed 
classrooms, cannot disadvantage those students in the same classrooms who are not identified as 
EL students. The present study of QuEST shows that such instructional improvements are not 
merely theoretically possible, they can be achieved in real school settings with actual Middle 
School science teachers. While much work remains to be done, to our knowledge this study 
represents the first such demonstration in a randomized controlled experiment that gains in 
content area knowledge in Science and gains in Vocabulary are possible for both ELL and 
English proficient students using a common approach to instruction that is designed to be 
optimal for the EL students. Of all research published between 1982 and 2009 in English in 
settings where English is the main medium of science instruction in elementary and secondary 
schools this is only published experimental study we know of that has found significant 
intervention effects for both ELLs and English proficient students in science knowledge and 
science vocabulary. 
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